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DIFFERENTIATED SERVICE NETWORK AND METHOD 
0F OPERATING A DIFFERENTIATED SERVICE NETWORK 

FIELn^EIHEJNVENI^ 

Th is invention reiates g— * a differentiated service networK and 

method of operating the network. 



^^res that service provide, such as ,SPs, offer 

^«*^-«-'*-'*--* d * r,,,-,d 

the iatency, throughput, ioss and jitter appiioatioh requi— . 

packet « and scheduiing in order to introduce d^erentiation - * 
ervice ,003, 0~— — — a„ow service probers to 
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bandwidth control mechanisms that treat different users differently. The IETF 
DiffServ Working Group has defined five classes of packet marking, namely the 
expedited forwarding class and the assured forwarding classes 1-4. These classes 
differentiate packets and flows, and while the relationship between the classes are 
not defined in detail, they are assumed to be specified by the network operator. 

The realization of differentiated service networks is somewhat ambivalent and 
problematic. It relies on general and common mechanisms for packet treatment. At 
the same time, operation of a differentiated service network re.ies heavily on correct 
provisioning of the network according to traffic requirements. This assumes that 
traffic requirements themselves are static and are not expected to change over time. 
Even with correct provisioning, networks cannot correct the on-the-fly packet 
marking and dynamically adjust to the new operating conditions in the network. 



Rl IMMARY OF THF INVENTION 

Embodiments of the present invention may provide a method of operating a 
differentiated service network having a plurality of routers. This may involve 
determining an operating condition at a first router and propagating an indication 
(i.e., a signal) of the operating condition at the first router to a second router. 

Embodiments of the present invention may provide a method of receiving 
indication of an operating condition at a first router and adjusting at least one 
parameter of a constraint or rule contained in the network profile based on the 
indication of the operating condition. 
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Embodiments of the present invention may provide a differentiated service 
networking including a firs, router and a second router coupied to the firs, router. 
The firs, rou,er may be associated with a flrs, entity (i.e., a QoS Firewail entity) to 
determine an operating condition at the first router. 

Other objects, advantages and salient features of the invention will become 
apparent from the following detailed description taken in conjunction with the 
annexed drawings, which disclose preferred embodiments of the invention. 

rriff DESCR IPTION OF THE DRAWINGS 

The foregoing and a better understanding of the present invention will 
become apparent from the following detailed description of example embodiments 
and the claims when read in connection with the accompanying drawings, all forming 
a par. of the disclosure of this invention. While the foregoing and following written 
and illustrated disclosure foouses on disclosing example embodiments of the 
invention, it should be understood that the same is by way of illustration and 
example only and is not limited thereto. The spirit and scope of the present 
invention being limited only by the terms of the appended claims. 

The present invention will be described with reference to the following 
drawings in which like reference numerals represent like elements and wherein: 
Figure 1 shows an implementation model of a differentiated service network; 
Figure 2 shows a differentiated service network according to an example 
embodiment of the present invention; 
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Figure 3 shows a stability graph for a two-class network; 

Figure 4 shows negotiation of user QoS requirements and system-level 

constraints; and 

Figure 5 shows a three-dimensional QoS parameter region. 

nFTAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Figure 1 shows an implementation model of differentiated services. This 
Figure shows the functional treatment of data packets as they enter and leave a 
switched fabric 20. Each of these functional modules may be performed by an edge 
router of the network. A classifier device 1 0 may examine inbound data packets 5 
and identify flows and associated performance parameters. A meter/marker device 
12 may measure properties selected by the classifier device 10 and mark packet 
headers according to the classification. A policer device 14 may monitor traffic flows 
to determine conformance with a bandwidth agreement and enforce the service level 
contracts. A queue selecter device 16 may queue packets according to their output 
classification and drop packets according to their discard policy. A scheduler device 
1 8 may schedule packets 1 5 for transmission on the outbound link in order to 
provide the level of service guaranteed by a service level contract. 

Figure 2 shows an example differentiated service network 20 according to an 
ample embodiment of the present invention. The network 20 may include three 

routers 22, 24 and 26, two edge routers 28 and 30, a bandwidth broker 40 and 
policy database 42. The routers may contain an operating system kernel to 
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support the functions to be performed. Other configurations and embodiments of 
the differentiated service network 20 are also within the scope of the present 
invention. 

The edge routers 28 and 30 may maintain the state of the traffic flow, enforce 
the policy of the traffic entering the network 20, map user requirements to the 
network resources and negotiate between user preferences and network 
capabilities. A typical chain of actions performed on input traffic may include the 
following: classification, metering, policy lookup and policing, shaping and marking. 
Output traffic may require information in order to control proper buffering and 
scheduling. 

Packet marking is important to the differentiated service network 20. For 
example, in case of the expedited forwarding class, the edge routers 28 and 30 may 
check the relevant traffic profile to verify that the required traffic flow fits the output 
aggregate specified for the class. In case of the assured forwarding classes 1 - 4, 
the data packets may be checked against the traffic profile and depending on the 
rate of the flow, the packets may be assigned a priority within the assured forwarding 
class. A flow complying with the traffic profile may receive the best treatment by 
being marked the highest priority. 

The network operator (e.g., an ISP) and customer relationship may be 
defined by a traffic profile, hereafter also referred to as a contract, policy or service 
level agreement. The profile may describe the rules and constraints for the packet 
marking. The rules list may be calculated with a classifier style approach or as a 
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linked list with flow attributes. The rules may be composed of a value statement and 
an action to be performed. The aggregation of the rules is an agreement between 
the customer and the operator that specifies the quality of service (i.e., the QoS) the 
customer requires and the cost associated with that service. 

As shown in Figure 2, each of the edge routers 28 and 30 and each of the 
core routers 22, 24 and 26 may include a QoS Firewall entity 23, 25, 27, 29 and 31 . 
The QoS Firewall entity may be a software implementation within each of the 
routers. The QoS Firewall entities may also be provided in a unit external to the 
router. The QoS Firewall entities may interface with any outside mechanism that 
wants to or tends to push QoS policy to the router. Each of the core routers 22, 24 
and 26 may also include a stability entity 44, 46 and 48 that may also be a software 
implementation to provide stability and fairness services. For ease of discussion, 
embodiments of the present invention may describe stability and fairness services 
as being part of the QoS Firewall entities although they may also be their own entity. 

Management of the traffic profile and its associated list of rules may be 
performed by the QoS Firewall entities 23, 25, 27, 29 and 31. The QoS Firewall 
entities 23, 25, 27, 29 and 31 may communicate with all interested parties regarding 
updates of the rule set. The QoS Firewall entities 23, 25, 27, 29 and 31 may also 
contain the management functionality to handle security, authentication, and 
translation of the policy request to accomplish reliable and correct operation of the 
differentiated service network 20. The QoS Firewall entities 23, 25, 27, 29 and 31 
may also manage buffers and schedule queue weights. 
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The QoS Firewall entities 23, 25, 27, 29 and 31 may also interface with policy 
servers and their proxy agents, map (i.e., translate) user requirements to 
system-level resources, dispatch notification, error and other events and mediate 
between static and dynamic input data. 

The core routers 22, 24 and 26 are provided in the interior of the network 20 
,o forward data packets according to their packet marking. For example, data 
packets may be forwarded from edge router 28 to core router 26 and then to edge 
router 30. In the core routers 22, 24 and 26, differentiation of traffic dasses may be 
realized through treatment of buffer queues and scheduling methods. 

The queue size for the differentiated service classes may influence the delay 
for the traffic flow. For example, the expedited folding class may be defined to 
have a minimum delay. In such a case, the expedited forwarding queue may be 
small compared to the queue of the other classes. The size of the assured 
forwarding class queues may be specified by the operator. Typicaliy, the assured 
forwarding Cass 1 will have a much shorter queue than the queue of the assured 
forwarding class 4. A threshold may be associated with each queue to indicate 

packet discard probability. 

The weight for the queue scheduling of the differentiated service classes may 
determine the available bandwidth. For example, the bandwidth of the expedited 
service class may be dependent on the weigh, associated with the expedited service 
queue. Allocation of bandwidth for the classes may also be operator dependent. 
Different weights may be allocated for every link and Cass in order to rea,ize a fair 
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and stable network. Fairness indicates fairfrom a service ieve, expectation and 
stable indicates the network wi,l be ab,e to operate correctly so tha, there is no 
prolonged starvation for less worthy traffic classes. The weight eolation may be 
similar to the physical link size and used as the primary parameter in provisioning 
the differentiated service network 20. 

The QoS Firewall entities 23, 25, 27, 29 and 31 may handle the weight 
management of queues and threshold management of the drop probability. The 
thresholds and weights may be communicated to the QoS Firewall entities 23, 25 
and 27 in the core routers 22, 24 and 26 from various controlling units, such as the 

bandwidth broker 40. 

The bandwidth broker 40 may negotiate the policy for traffic flows entering the 

network 20 through the edge routers 28 and 30. The bandwidth broker 40 may be 
responsible for proper allocation of traffic in the network 20. Accuracy may be 
increased by the bandwidth broker 40 collecting the network topology through query 
of the routing tables of the different nodes in the network 20. The bandwidth broker 
40 may maximize the policies accepted while still guaranteeing a fair and stable 
network. 

The bandwidth broker 40 preferably is an agent responsible for allocating 
preferred service to users and for configuring the routers 22, 24, 26, 28 and 30 with 
the correct forwarding behavior for the defined service. A policy database 42 may 
be connected to the bandwidth broker 40 and contain information on how users, sets 
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addition, the database 42 may contain Monition to authenticate requesters. 

The bandwidth broker 40 may reiy on heuristical vaiues and rules se, by the 

drop probability thresholds may also be determined by the values set by the 
operator. The bandwidth broker 40 may obtain feedback from the routers 22, 24, 
26, 28 and 30 about the trafficflow conditions for each of the traffic classes and for 
every link. For example, Figure 2 shows communication signals 32, 34, 36, 38 and 
39 between each of the routers and the bandwidth broker 40. The signals 32, 34, 
36 38 and 40 may contain the feedback information. More specifically, the 
communication signal 32 may be transmitted between the bandwidth broker 40 and 
the edge router 28. The communication signal 34 may be transmitted between the 
bandwidth broker 40 and the core router 34. The communication signal 36 may be 
transmitted between the bandwidth broker 40 and the core router 26. The 
communication signal 38 may be transmitted between the bandwidth broker 40 and 
the core router 24. The communication signal 39 may be transmitted between the 
bandwidth broker 40 and the edge router 30. Other types of signais and methods of 
transmitting signals are also within the scope of the present invention. 

Rather than identifying the individual flows or packets metric (e.g. packet loss 
rate delay, etc.), the bandwidth broker 40 may map the quantitative values into 
qualitative indications. This may be accomplished using a signal corresponding to a 
state parameter such as a network traffic status. This state parameter may be 
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represented by a color such as red, green or yellow and may be communicated 
rather than relaying the packet loss rate for a given traffic class. A different number 
of state parameters may also be communicated in accordance with the present 

invention. 

in order to calculate the quantitative indications, the network 20 may utilize 
packet loss as an indication for reactive actions. Average service of the given traffic 
classes and their associated flows may be used to calculate the quantitative 
indication. Fair and stable network parameters may be used for guidelines. 

Stable operation of average traffic through a router is desirable, and this 
stability may take the value of the different traffic classes into account. Therefore, in 
accordance with the present invention, a calculation may be performed by the QoS 
Firewall entity in every router to set the scheduling weight and to communicate the 
status such as a signal corresponding to red, green or yellow. A signal 
corresponding to this color indication may be propagated to the edge routers 28 and 

30 and the bandwidth broker 40. 

Stability of the network 20 may be dependent on scheduling decisions made 
at the router level. There are two notions for stability. First is router overflow when 
the number of packets coming to the network 20 is too high with respect to the 
processing capability of the router. Despite the order the data packets are 
processed, the incoming workload may exceed the capacity of the individual router. 
Second is network instability when higher precedence is given to certain classes so 
as to starve some of the router's traffic and create blocking or oscillatory modes. 
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A randomized schedule with weights depending on the precedence level of 
the class may be provided such that when router stability is satisfied, then so is the 
network stability. Randomness may break the blocking patterns and the oscillation 
modes. Furthermore, if the weights are fixed with respect to a QoS agreement then 
stability regions may be defined. For example, the assured forwarding class 1 may 
receive a service probability twice that of the assured forwarding class 2, which may 
receive a service probability twice that of the assured forwarding class 3, which may 
receive a service probability twice that of the assured forwarding class 4. 

A stability region may be a relation between the different flow rates for the 
respective QoS classes. These stability regions may provide a qualitative indication 
on the stability of the network 20. As will be discussed below, the routers 22, 24, 26, 
28 and 30 may compute the flow of the incoming classes, and check which area of 
its processing domain it is, and then issue an indication such as a signal 
corresponding to one of the colors of green, yellow or red. When these indication 
signals are received by the edge routers 28 and 30 and/or the bandwidth broker 40 
then a decision may be made regarding the traffic flow. 

Figure 3 shows a stability graph for a two-class network (i.e., A, and A 2 ). 
Based on a stability calculation that will be described below, if the result of the 
calculation falls within the unstable area US as shown on the graph, then a red 
indicator may be provided symbolizing a congestion state. If the calculation falls 
within the area RS as shown on the graph, then a yellow indicator may be provided. 
Finally, if the calculation falls within the area NS as shown on the graph, then a 
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green indicator may be provided. The present invention is also applicable to 
indicators other than red, yellow and green. 

The greater the network stability (NS) domain, the more efficient the policy. 
The network stability domain may be the domain where higher priority may be given 
to trickling flows without threatening the overall stability of the network 20. The 
network stability may be given as the waiting time of packets in a router. The 
scheduling weights and scheduling decisions may be computed at each of the 
routers 22, 24, 26, 28 and 30 and be distributed independent of the network 
topology. The information to make decisions may be acquired on line and may be 
adaptive to long-term tramc fluctuations and network topology changes. 

As discussed above, each class may correspond to a level of service in which 
each class is treated differently according to an agreement between the customer 
and the service provider. However, due to bandwidth constraints on links between 
routers, some data packets may have to wait in the individual router before they are 
propagated to the next router. The data packets may be ordered and scheduled in 
the output buffers of the respective routers. However, the buffers may overflow due 
t0 business of the sources, or business created by the network-wide interactions 

between the different flows. 

A randomized scheduling algorithm may pick packets according to a 
distribution to prevent the burs.iness and ensure long term stability over the network. 
The distribution may be computed according to the value of the traffic rates. In order 
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t0 ensure flow conservation, .he distribution may be picked to give more weigh, to 
flows having a greater incoming packets arrival rate. 

One condition for stability may be that the average traffic on a link may be 

,o be stable. The load on the link should be less than one (i.e., p < 1) for this 

process a packet divided by its interarrival time. 

The four levels of the assured forwarding classes 1-4 may share the 
bandwidth Ml after processing the expedited folding class. For example, 
assume the network knows how many packets of the assured folding class 1 
have to be processed for each packet of the assured forwarding class 4. Then, 
when congestion occurs, the network may assume that on average for a. packets of 
th e assured forwarding class 4, the network may serve a 3 packets of the assured 
forwarding class 3, a 2 packets of the assured Warding class 2, and a, packets of 
, he assured forwarding class 1. The higher the ratio a,/a 4 , the higher the 
precedence of the assured fora/arding class 1 

The ratios may accommodate traffic with incoming flows such that V*« = a,/* 
and .A = where X, is the arrive, rate (i.e., the inverse of the mean interarrival 
time). This together with the necessary condition for stability p < 1 may define a 

stability hyperplan. 

The effective load at a given level of QoS is the quantity that takes into 
account the worst case effect of the other levels of service. For example, a data 
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This may be reflected in the effective load of this class. 

The randomized scheduling policy may have a Poisson-.ike behavior. The 
effective load of class i on a fixed link may be given by: 



(1) 

p, e = (a, + I a/aPW 

j=1 ,2,3,4;H 



wh ere a, is the average <ime i. ta.es for the f,xed iink to process a pacKe, o, Cass , 

and A, is the arrival rate. 

Based on equation (1 ). the effective ,oad may be proportions to the arriva, 
rate and the interaction between different ciasses may be expressed by the p V s in 
the definition of p* . It is desirable that p e ■= 1 for all classes and all nodes to ensure 

the stability of the network. 

An example algorithm that computes the effective load on each link will now 
be described. Incoming streams at a router may be sorted by their destination port, 
and sen, to the respective output buyers. These streams may be then scheduled to 
be sen. on their destination link according to their priority leve,. The following 
algorithm may be provided a, the output buffer to evaluate the quantities that it 
needs (i.e., the packets' lengths and the arriva, rates). From these quantities, ma 

a flag may be sen. to the bandwidth broker 40 providing a distance from the 
unstable region. The .asks may .hen be scheduled so .hey are sen. .o me ou.pu. 
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link according t o their priority level. The algorithm may therefore perform an 
eva ,ua„on procedure, a computation procedure and a scheduling procedure as - 
now be described. 

During the evasion procedure, the router may tracK the arrivai rate o, each 
class (arr.ratetclass,,, the mean service time of each ciass (srv^Cass,, and the 
arrive, time of the last paoket of each Cass (arrjimetclass]). 

Upon the arrivai of a new packet, the algorithm may update these quantities 

as follow: 

arr ra.e [ class,-1/(p1(.ime-arr_«ime[ciass l ) + (1 -pD/arr.rate^iass]) (2) 
s^ime^ass^tpUength/iin^rateWI-P^srvJimelCass) (3) 

arr_time[class]=time. (4) 

The parameters p1 and P 2 are quantities between 0 and 1 . Asmallpf orp2 

may converge more slowly to the mean value. 

During the computation procedure, the effective loads may be computed w„h 
th e following iterative steps using the vaiues o, arr.rate.iass, and srv.time.lass,. 
The effective load values may be stored in the array rho[class]. 
. initialization. rho[i] = 0 for all classes i. 
. iteration. rho[i) = (srvJimeH ♦ E a^rhoffl srvjimelffiarr.rateli]. 
For the assured forwarding Cass 1 , for example, rho,AF„ = (srv.time^l, + 
a(AF2)/a(AF1) rho,AF2, srv_«me(AF2) + ... + a(AF4ya)AF1) rho^srv.time^, 
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The global stability condition may be computed as follows: rho[link] =£ srv_time[i] 
arr-rate[i]. 

If the values of arr_rate[class] and srv_time[class] converge to a constant, 
then so will rho. Also, rho may converge increasingly starting from 0. 

For the network to be stable, p[i] < 1 for all classes and all routers. The 
distance from p[i] to 1 may be the indication of congestion. Congestion may occur if 

only one of the p[i] is more than 1 . 

Each router 22, 24, 26, 28 and 30 may send to the bandwidth broker 40 either 
the distance from one for the effective load of each class on each link, or a signal 
indicating a network traffic status. The network traffic status may be represented by 
a coloring scheme that packetizes the distances into an indication of congestion. 
The core routers 22, 24 and 26 may also forward this information to the edge routers 
28 and 30. 

One example embodiment may use the following coloring scheme. If all p[i] < 
1 , then the color indication may be green. Green may represent stability. The color 
indication may be yellow when the necessary stability condition is satisfied, namely 
p[i] < 1 , but at least one of the rho [i] is more than 1 . Yellow may represent an 
indication between stable and unstable. The color indication may be red when p[i] > 
1. Red may represent unstable. Other colors, indications and methods of 
determining the degree of stability and unstability are also within the scope of the 
present invention. 
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The functional modules that implement the QoS Firewall entities 23, 25, 27, 
29 and 31 and its mode, of service sellers (providers) and buyers (customers) will 
now be described in greater detail. 

The QoS Firewall entities 23, 25, 27, 29 and 31 may perform several 
functions including but not limited to: (1 ) interface with policy servers and their proxy 
agents; (2) make admission control decisions; (3) capture user QoS requirements; 
(4) create store, retrieve, and modify se^ice-level contracts (profiles); (5) negotiate 
between user requests and system-level constraints; (6) make QoS trade-off 
decisions; (7) map requirements to system-level resources according to a 
pre-described set of rules; (8) monitor QoS service levels; (9) dispatch notification, 
error and other events; (10) mediate between static and dynamic input data; and 
(11) provide locking on data being modified. 

The QoS Firewall entities 23, 25, 27, 29 and 31 may also interface with the 
several system components, including but not limited to QoS client(s), other QoS 
Firewall entities, OS kernel and policy servers, policy proxies, etc. 

One or more autonomous service agents may communicate and cooperate 
with each other via a message bus to support QoS configuration, negotiation and 
monitoring. The service agents (e.g. billing, negotiation, monitoring, etc.) may exist 
on different platforms linked by communication channels. Service agents are 
considered 'sellers' or 'providers' of a particular service, and clients are considered 
■buyers' or 'customers' of the service. There may be no restrictions or constraints as 
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may exist on every end-system or every node in the network 20. 

The union of ail the QoS service agents is equivalent to a single QoS Firewall 
entity that supports mapping of QoS parameters from application leve, to system 
,eve, negotiation o, QoS parameters, reservation o, resources in the system and 
.acting to QoS fluctuation, That is, the QoS Firewall entities 23, 25, 27, 29 and 31 
may be a functional aggregate of one or more QoS service agents. 

,„ ord er to realize the QoS service providers and their customers the following 
.unction modules may be provided within the entity: d ) a user interface module; (2) 
a mapping module; (3) a reservation module; (4) a monitoring module; (5) an 
adaptation/negotiation module; and (6) an inter/intra communication module. 

The user interface module may provide the means to describe lists of 
parameters, which are in the form o, name/value pairs. The mapping module may 
translate user-level parameters into a set of system leve, parameters (e.g., host and 
network parameters). The reservation module may reserve system/host/network 
resources needed by a user customer according to their requirements. The 
monitoring module may control and measure watermark levels, and in the case o, 
violations, send an alert message to the adaptation module that executes local 
mechanisms for violation recovery. The adaptation/negotiation module may execute 
mechanisms that attempt to correct network/system degradation. In addition, this 
module may handle negotiations between users' QoS parameter requests and 
system-level constraints. 
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The in,er/in,ra communication module may implement reiiabie communication 
channeis between QoS Firewai, entities, associated agents and their customers. 
This moduie may aiso define the communication endpoihts (seiiers and buyers) and 
the services between the endpoints. For example, the protocol to communicate Wh 

a signaling daemon in order to reserve network resources may be spewed in «s 

module. 

in addition, the inter/intra communication module may implement the 
m essage bus facility that provides flexible methods for service providers and 
consumers to communicate with one another and share data. Requests may be 
messages sen, to a component to request it perform some action, events may be 
usages tha, are broadcast by a provider and received by consumers. Consumers 
may subscribe to the classes o, events they wan. to receive, and the message bus 
may keep track of the event subscription. 

Any QoS API that defines the request/reply protocol stream between a QoS 
service and its customers may be implemented on top o, the communication module 
and its associated sub-moduies. In other words, these functional modules are the 

lowest in the QoS framework. 

These functional modules may provide an infrastructure for a simple 
code-exchange and data-exchange based system. Data-exchange signifies that the 
,ogic o, an client application is statically installed and coordination with a server 
application is accomplished by exchanging data messages according to a 
predefined protocol. Code-exchange may mean that coordination between dents 
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and servers is achieved by sending around code fragments that alter the data 
instances inside the network's hosts. The resuiting system may be a combination of 
both modeis whereby code-exchange will take place using an interpreted language 
between service providers and customers, and classical data-exchange techniques 
may be used in data streams between service providers. 

Embodiments of the present invention may provide unique features and 
advantages. The edge-style composition and propagation of network status 
information may provide a scalable and robust solution. Inference in the routers 
and QOS Firewall entities 23, 25, 27, 29 and 31 may provide fallback mechanisms 
when a bandwidth broker 40 is unavai,able. The stability calculation may be a 
refined and cealesced monitoring value that simplifies network management. The 
feedback mechanism may be achieved since QoS Firewall entities 23, 25, 27, 29 
and 31 may communicate with the edge routers 28 and 30 or the bandwidth broker 
40. The edge routers 28 and 30 or the bandwidth broker 40 may dynamically set 
profiles or network node parameters to comply with real network situations based on 
the information they receive from the feedback mechanism. 

A QoS negotiation and renegotiation procedure according to an example 
embodiment o, the present invention will now be described. A QoS parameter value 
may change during the lifetime of a connection. That is, once negotiated, in 
accordance with the present invention a QoS parameter value may be renegotiated. 
Thus, the network 20 may dynamically adjust the QoS of a live connection or flow 
without requiring disconnectfreconnect. For example, the edge router 28 may 
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reoei ve an indication of an operating condition such as a signal corresponding to the 
stability of a core router. The QoS Firewall entity 29 of the edge router 28 may then 
adjust a parameter of a constrain, in the network profile based on the indiction o, the 
operating condition. The QoS Firewall entity 29 may renegotiate the constrain, or 
may make a recommendation to the network operator on changing the profile. 

The network may specify a set of prioritized QoS parameters, including cost 
metrics. The QoS Firewall entities 23, 25, 27, 29 and 31 may contain the 
intel.igence and logic necessary to perform QoS parameter mapping and resource 
usage calculation. Addition* the QoS Firewal, entities 23, 25, 27, 29 and 31 may 
contain the intelligence and logic necessary to make trade-off decisions based on 
the importance of the QoS parameters, customer priority ranking and costs 
associated with meeting those parameters. In one example embodiment, the user 
may specify a plurality of constraints in order of importance, degradation of 
service occurs, then the QoS Firewall entity may operate to select the highest 
constrain, that does no. result in degradation of se^ice. The QoS Firewa,, entity 
may also renegotiate with the underlying router to determine » the changed 
parameters can be accommodated. This may be done without cutting off service to 

the traffic flow. 

The above described embodiments discussed the mechanisms to achieve 
QoS negotiation. The following will describe how initia, QoS requirements may be 
realized and describe what happens when the QoS Firewall entities 23, 25, 27, 29 
and 31 receive the co,or signal indicating that some action should be performed due 
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t0 adverse conditions in the network. U.S. Patent Application No. (Attorney Docket 
No. 730.38193X00) entitled "Method And Network For Propagating Status 
information", the subject matter of which is incorporated herein by reference, 
discloses example embodiments of how signals (such as color indications) may be 

propagated through the network. 

QoS requirements may be conveyed from the QoS client to the QoS Firewall 
entities 23, 25, 27, 29 and 31 . The level of granularity may range from the broad to 
the specific. This is, all inbound protocol-X traffic may have a minimum rate of nnn. 
Inbound HTTP traffic from the network prefix xxx. xx to the destination 
xxx.xxx.xxx.xxx may have a rate of nnn between the hours of 9am and 5pm. 

The QoS specifications may encompass the following categories: (1) 
expected performance characteristics to establish resource commitments; (2) degree 
of resource commitment to maintain performance guarantees; (3) price a user is 
willing to incur to obtain a level of service; and (4) degree of adaptation that can be 
tolerated and the scaling actions to be taken in the event the contracted service 
cannot be met. These categories are one example embodiment of the present 
invention as other categories are also within the scope of the present invention. 

An example embodiment of QoS negotiation procedure will now be described. 
The QoS requirements may be assessed/evaluated to determine if they can be met. 
For example, if the requested level of service cannot be provided, then a period of 
negotiation may occur where the user is asked what level of degradation is 
acceptable. This may be an iterative process that takes place between the QoS 
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client 



, the QoS Firewall entities 23, 25, 27, 29 and 31 and the underlying system 



platform. 

The iterative process may involve determining the following: (1 ) the type of 
agreement that is to be reached (e.g. target, range, upper limit, lower limit, threshold, 
etc.); (2) whether resources are to be allocated to the activity; (3) whether the QoS 
achieved is to be monitored; and (4) the action to be performed if and when the 
agreed QoS cannot be maintained. The actions may include renegotiation, reducing 
the level of service, reducing another competing activity of lower precedence and/or 
assigning a predefined penalty. Figure 4 shows one example embodiment of 
negotiation of user QoS requirements and system-level constraints. 

A QoS enforcement procedure will now be described. The QoS parameters 
may be monitored and system-level resources allocated/reallocated or some other 
action performed in response to system anomalies. System resources under 
consideration may include: reserved buffer space, reserved queues, link bandwidth, 
CPU utilization and timeslice allocation. The QoS Firewall entities 23, 25, 27, 29 
and 31 may be responsible for monitoring these resources in order to detect 
deviations in the QoS parameters. 

When there is a state change, then resource adjustments can be 
automatically handled by the QoS Firewall entities 23, 25, 27, 29 and 31 when 
fallback requirements have been previously specified along with actions to take 
when the system state changes (i.e., QoS degradation is detected). This allows the 
QoS Firewall entities 23, 25, 27, 29 and 31 to gracefully and transparently degrade 
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QoS parameters under certain conditions. Renegotiation is one action that may be 
taken. This will be discussed below in detail. 

When there is a state change and resource adjustments cannot be 
automatically made in order to compensate (as speeded in the original agreement), 
then the application level may be notified. The operator can either adapt to the new 
level of QoS or scale to a reduced level of service. 

For both QoS negotation and renegotiation, a list of QoS vectors may be 
used. This lis, may define the desired quality and may be sorted by preference or 
importance (e.g. the parameters are weighted). In the simplest case, three axes (x, 
y, z) that represent the QoS parameters delay, throughput and rate can be used. 
The intersection of their bounded region defines a QoS parameter region. 

The QoS of a traffic flow may be represented by the vector Q[d,r,t] where d 
corresponds to the delay, r corresponds to the rate and t corresponds to the 
throughput. The difference between rate and throughput is packet loss. 

During negotiation, a list of vectors may be ranked in order of importance 

from 0 to a predefined maximum N such as: QoS=(QoS 0 QoS N ). The ranking 

may be operator and customer specific. During the process of negotiation, the 
processing flow may be from the most important vector to the least important vector. 

Parameters in each vector may be multi-dimensional. Possible elements for 
each parameter may include the following: maximum value, maximum rate, percent 
increment/decrement, importance and probability. 
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As discussed above, .he QoS parameter values for a live connection may be 
changed during QoS renego.ia.ion. The II, o, QoS veCors may be predefined. 
Figu re 5 shows a .hree-dimensiona, QoS parameter region. This region may be 
b ound to .he ranges specified by .he QoS parameters along .he x, y and z axes. 
During nego,ia.ion, .he processing flow may be bidirediona, meaning .ha. start,ng 
with .he QoS vector of a iive fiow or connection (i.e.. the current working vector), 

0 processing can proceed in both directions towards .he most important vector and 

1 towards the least important vector. 

I ln order to reduce the rego.ia.ion process, QoS vectors may be categorized 

| inteciassesorgroupstoenablereaiizationof.heQoSparame.ers.obesp.itamong 

I more than one vector. For example, i, there are three vectors in a group, then QoS 
I negotiation may consider a„ o, .hem a. ,he time and selec, from a se, of three for 
1 each o, the d, r, and , parameters in order «o sa.isfy .he QoS requirement 

Customers' QoS requirements may be mapped to the QoS vectors, and each 
15 customer may have a priority ranking. Ranking may be based on the customer, 
ability and willingness to pay for service. 

An aigorithm and mechanism may be provided whereby QoS resources can 

be bought and sold from one customer to the next. 

Figure 5 also shows a negotiation space in which negotiation is possible as 
20 tn eQoSvec.orassocia.edwi,hacurren,connec.ionor fl ow. This vector may be 

a ,tered using QoS renegotiate iimi.ed to the range boundaries defined by the QoS 
parameter region. 
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Cos, metric may be an addition, parame.er used in the subseguen. pro,oooi 
and algorithms. Cos, metric may inciude the following values: b ase monetary rate, 
percentage increase increment, — monetary ra,e, importance factor, and 
probability. 

B0 ,h ,he initia, nego,ia,ion process and renego,ia,ion process may utilize ,he 
sa mese, of a,gori,hms and protocols. However. , he scenarios for , hem differ. The 
scenario for negotiation wili firs, be described Mowed by ,he scenario for 

renegotiation. 

The network operator may specify QoS parameters including a lis. o, 
prioritized fallback resources and actions that may be performed in ,he even, of 
service degradafion. The QoS Firewall entities map (i.e., ,rans,a,e) ,he parameters 
t0 system resources. Through the negotiation processing, ,he QoS Firewall entities 

accepted, the parameters may be successfully realized and applied ,o the system. If 
rejected, the operator may be notifed. On the other hand, the scenario for 
.negotiation is as follows. Through monitoring, the QoS Firewal, entity may 
oetermine that one or more connections are experiencing service de g rada„on. 

» the available system resources can satisfy one o, the available remaining QoS 
vectors. If ifcan be met, .hentha, vector becomes associa,ed w,th the connection 
and the netfconnection, if any, reguiring renegotiation is handled, lfi.canno.be 
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met then the operator may be no.if.ed and the resources a„oca«ed for the QoS 
sector are released. The next connection, if any, retiring renegotiation m ay then 
be handled. 

A n example embodiment of a negotiation and renegotiation algorithm wil, now 
be described. Other embodiments o, the aigorithm are aiso within the scope of the 
present invention. The following are the algorithm definitions: 

• Number of flow requiring attention. A 

. Flows to be processed, sorted in order to priority: Process = (Process, 
Process A ) 

•Maximum number of QoS vectors per flow: N 

. vector of weighted QoS parameters per flow: QoS = (QoS 0 .... QoS N ) 

. vector of allocated QoS resources: QoS Allocated 

. Maximum system resources available for QoS parameters: QoSMax 

. QoS mapped to QoS parameter vector QoSRequest = (QoS 0 , .... QoS A ) 

The following are the algorithm initializations: 

• QoSRemaining = SoSMax 

• ACounter = A 
•NCounter = N 

The negotiation algorithm may be as follows: 
for (j = 0 ...NCounter) { 

QoSRequest' = [QoSj]; 

if (QoSRemaining -QoSRequest, <=QoSMAX){exit;} 
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else {QoS Allocate^ = QoSRequesti} 

}• 

The renegotiation algorithm may be as follows, 
while (i=1 ... ACounter){ 

for 0 = 0 NCounter){ 

QoSRequest k = Process^QoSj]; 



■ onqRpauest k <= QoSMAX){exit;} 
i^QoSRemaining-" QobKequesi « 

else { QoS Allocated, = QoSRequest k } 



{ 

An additional .eve. of detail for the two algorithms may be provided, .t is at 
this point in the algorithm processing whereby the N-dimensiona. QoS vectors, 
[Q o Sj] and Proces, fl** may be mapped to system .eve. resources. The cost 

cost maximum matching. Since a N-dimensiona, vector may be used to express 
the required QoS parameters, a cost function C(d, r,t) may be provided where d 
ignifies de.ay, r signifies rate and t signifies throughput. 
The following may be the cost metric definitions: 

• Base monetary rate: Base Rate 

• percentage incremented: Incr 
« Maximum monetary rate: MaxRate 



a 



si 
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• Importance of cost: ImpCost 

• importance of QoS: ImpQoS 

• Current cost: Cost 

The following may be the cost metric initializations: 

• Cost = Base Rate 

The algorithm may then be as follows. 

while (i = 1 ... ACounter){ 

if (Cost<MaxRate&&lmpCost < ImpQoS) { 

Cost = Cost + (Incr * Cost + 100); 

} 

i++; 

}• 

Accordingly, embodiments of the present invention may provide a method of 
operating a differentiated service network having a plurality of routers. This may 
involve determining an operating condition at a first router and propagating an 
indication of the first operating condition from the first router to a second router. 
Embodiments of the present invention may also provide a method of operating a 
differentiated se-vice network by receiving an indication of an operating condition 
and adjusting at least one parameter of a constraint in a network profile based on 
the indication of the operating condition. 

While the invention has been described with reference to specific 
embodiments, the description of the specific embodiments is il.ustrative only and is 
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not to be construed as limiting the scope of the invention. Various other 
modifications and changes may occur to those skilled in the art without departing 
from the spirit and scope of the invention. 
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